A three-dimensional (3D) morphable model of a user's face may be generated to enable virtual augmentation of the user's face. For example, the 3D model of the user's face may be used to virtually try on various facial accessories, such as glasses and jewelry, to save the user a physical trip to a retail store. It would be desirable to efficiently generate a 3D morphable model of a user's face that closely fits the user's actual face.